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DETAILED ACTION 



Response to Request for Reconsideration 

This Office Action is in response to applicant's request for reconsideration filed on 4/28/2004. 



1. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

2. Claims 1-7, 11-12, 16-19 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Hirata et al. U.S. Patent No. 6,240,423 (hereinafter Hirata). 

3. Claim 1: 

Hirata teaches a method of visualizing and retrieving a data file comprising: 
Displaying a plurality of images representing corresponding data files on a display device 
using a first distance metric (Calculating a first similarity between a query image and images in 
the database using a region-based matching to produce a first set of similar images. Images being 
retrieved in order based upon the distance between the query image and the result candidate 
images, column 6) between each data file (e.g., Based on the mutual similarities between the 
images, retrieval results of the candidate images are sorted. The images which are similar to each 



Claim Rejections - 35 USC §102 
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other are assumed to be under one group and are re-ordered based on the similarity and users can 
specify the forms of the output for display; figures 1-11; column 6-9); 

Redisplaying a portion of the images on the display device using a refined distance metric 
(e.g., Hirata teaches using a distance metric such as a second similarity to retrieve images based 
on the set of similar images retrieved using a first similarity. The second similarity is now a 
"refined" distance metric because it refines the search for images. The second similarity of 
Hirata thus meets the claim limitation of "a refined distance metric." Here, a refined set of 
similar images is produced from the first set of similar images produced using a first distance 
metric, figure 9, 1 1 and 12; column 6-9); and 

Performing at least one of retrieving, marking, and selecting at least one desired data file 
(Hirata teaches retrieving and selecting at least one desired data file; see figures 1-11; column 6- 
9). 

Claim 2: 

The claim 2 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of repeating the redisplaying step until a desired data file is identifiable. 
However, Hirata further discloses the claimed limitation of repeating the redisplaying step until a 
desired data file is identifiable (e.g., column 4 wherein Hirata teaches specifying a hierarchical 
structure using the authoring tools in which a user may repeat the step of specifying regions of 
the query image until a desired data file is identifiable). 

Claim 3: 
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The claim 3 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of computing a feature vector for each data file and calculating the first 
distance metric between each data file using a first subset of data contained in the feature vector. 
However, Hirata further discloses the claimed limitation of computing a feature vector for each 
data file (e.g., similarity is determined based on a number of features derived from the respective 
images; column 1) and calculating the first distance metric between each data file using a first 
subset of data contained in the feature vector (e.g., column 6-9). 

Claim 4: 

The claim 4 encompasses the same scope of invention as that of claim 3 except additional 
claimed limitation of calculating a second distance metric between each data file using a second 
subset of data contained in the feature vector which is greater than the first subset. However, 
Hirata further discloses the claimed limitation of calculating a second distance metric between 
each data file using a second subset of data contained in the feature vector which is greater than 
the first subset (e.g., column 6-9). 

Claim 5: 

The claim 5 encompasses the same scope of invention as that of claim 4 except additional 
claimed limitation of computing the feature vector for each data file before starting the method; 
storing the feature vector for each data file; and accessing the feature vector for each data file. 
However, Hirata further discloses the claimed limitation of computing the feature vector for each 
data file before starting the method; storing the feature vector for each data file; and accessing 
the feature vector for each data file (e.g., grouping the second set of similar images into clusters 
based upon a similarity calculation among members of the second set of similar images and re- 
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ordering the second set of similar images based upon a similarity calculation among members of 
the second set of similar images wherein feature vector calculation is involved for each data file 
and the results are stored in the database; see column 9). 
Claim 6: 

The claim 6 encompasses the same scope of invention as that of claim 4 except additional 
claimed limitation of each feature vector having a length at least eight. 

However, Hirata further discloses the claimed limitation of each feature vector having a length at 
least eight (e.g., considering the multiple features with respect to the image regions). 
Claim 7: 

The claim 7 encompasses the same scope of invention as that of claim 4 except additional 
claimed limitation of the feature vector including at least one of a color feature and a texture 
feature. However, Hirata further discloses the claimed limitation of the feature vector including 
at least one of a color feature and a texture feature (e.g., figures 1-11; column 2). 

Claim 11: 

The claim 1 1 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of the data files being image files. However, Hirata further 
discloses the claimed limitation of the data files being image files (e.g., figures 1-11; column 5). 

Claim 12: 

The claim 12 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of the data files being video files. However, Hirata further discloses 
the claimed limitation of the data files being video files (e.g., figures 1-1 1; column 5). 
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Claim 16: 

The claim 16 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of the portion of the images redisplayed being graphically selected 
by the user. However, Hirata further discloses the claimed limitation of the portion of the images 
redisplayed being graphically selected by the user (e.g., figures 1-11; column 4). 



4. Claim 17: 

Hirata teaches a method of interactively retrieving a data file from a set of data files in 
real time comprising: 

Displaying a plurality of images, each image corresponding to a data file, on a display 
device using a first distance metric between each data file (e.g., Based on the mutual similarities 
between the images, retrieval results of the candidate images are sorted. The images similar to 
each other are assumed to be within one group and are re-ordered based on the similarity and 
users can specify the forms of the output for display. Clustering image data files is done using a 
distance metric such as a first similarity; figures 1-11; column 6-9); 

Interactively selecting, by a user, a portion of the images (e.g., a user specify the regions 
of the images; e.g., the region division of figure 3; figures 1-11; column 6-9); 

Redisplaying the portion of the images in real time on the display device using a refined 
distance metric (e.g., Hirata teaches using a distance metric such as a second similarity to 
retrieve images based on the set of similar images retrieved using a first similarity. The second 
similarity is now a "refined" distance metric because it refines the search for images. The second 
similarity of Hirata thus meets the claim limitation of "a refined distance metric." Here, a refined 
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set of similar images is produced from the first set of similar images produced using a first 
distance metric. Re-order is based on the similarity among the candidate images of figure 9 or 2 nd 
stage image matching is based on boundary of figure 1 1 and grouping and re-ordering is based 
on the similarity among candidates using the refined distance metric of figure 11; figures 1-11; 
column 4); and 

Retrieving a desired data file (retrieved data file is displayed on a display device; figures 
1-11; column 4). 



Claim 18: 

The claim 18 encompasses the same scope of invention as that of claim 17 except 
additional claimed limitation of computing a feature vector for each data file and calculating the 
first distance metric between each data file using a first subset of data contained in the feature 
vector. However, Hirata further discloses the claimed limitation of computing a feature vector 
(visual attributes values, including color, shape and texture-related values; see column 2) for 
each data file and calculating the first distance metric between each data file using a first subset 
of data contained in the feature vector (a subset of data refers to the image region and Hirata 
teaches extracting the feature vector corresponding to each image region in the feature-based 
image processing; figures 1-11; column 6-9). 

Claim 19: 

The claim 19 encompasses the same scope of invention as that of claim 18 except 
additional claimed limitation of calculating a second distance metric between each data file using 
a second subset of data contained in the feature vector which is greater than the first subset. 
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However, Hirata further discloses the claimed limitation of calculating a second distance metric 
between each data file using a second subset of data contained in the feature vector which is 
greater than the first subset (e.g., re-ordering the output images using a second distance metric; 
figures 1-11; column 6-9). 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 8, and 13-14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hirata et al. U.S. Patent No. 6,240,423 (hereinafter Hirata), as applied to claims 1 and 17 above, 
and further in view of Moghaddam et al. U.S. Patent No. 6,584,221 (hereinafter Moghaddam). 

7. Claim 8: 

(1) The claim 8 encompasses the same scope of invention as that of claim 4 except 
additional claimed limitation of the feature vector including at least one of a color histogram, 
color moment, color coherence histogram, Multiresolution Simultaneous Augoregressive 
(MRSAR) Model, coarseness, and directionality. 

(2) Hirata is silent on the claimed limitation of the feature vector including at least one of 
a color histogram, color moment, color coherence histogram, Multiresolution Simultaneous 
Augoregressive (MRSAR) Model, coarseness, and directionality. 
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(3) Moghaddam discloses the claimed limitation of the feature vector including at least 
one of a color histogram, color moment, color coherence histogram, Multiresolution 
Simultaneous Augoregressive (MRSAR) Model, coarseness, and directionality (e.g., 
Moghaddam column 3, lines 2-10; column 3, lines 64-67; column 4, lines 1-8). 

(4) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Moghaddam 's feature vectors into Hirata's method of visualizing and retrieving an image from 
the database because Hirata suggests visual and semantic features including color, shape and 
texture-related values (Hirata column 2). 

(5) One having the ordinary skill in the art would have been motivated to do this because 
it would have advantageously provided the various image features for accurate image extraction 
and matching. 

Claims 13-14: 

(1) The claim 13 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of establishing a fixed scale that spans a maximum distance 
between the plurality of data files; and indicating a relative position on the fixed scale for the 
redisplay of the portion of the image, thereby providing the user with a reference frame. 
The claim 14 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of the fixed scale being at least one of a linear scale, a logarithmic scale, and a 
hyperbolic scale. 

(2) Hirata is silent on the linear scale. 

(3) Moghaddam discloses the linear scale (e.g., Moghaddam column 4). 
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(4) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Moghaddam's linear scale into Hirata's method of visualizing and retrieving an image from the 
database because Hirata suggests specifying image regions (Hirata column 2). 

(5) One having the ordinary skill in the art would have been motivated to do this because 
it would have advantageously selected image regions at different resolutions. 

8. Claim 9, 10, and 20 are rejected under 35 U.S.C 103(a) as being unpatentable over 
Hirata et al. U.S. Patent No. 6,240,423 (hereinafter Hirata), as applied to claims 1 and 17 above, 
and further in view of Jain U.S. Patent No. 6,121,969 (hereinafter Jain). 

9. Claims 9, 10, and 20: 

(1) Hirata teaches a method of visualizing and retrieving a data file from a set of data 

files. 

(2) Hirata is silent on the claimed limitation that the first distance metrics are mapped 
into an N-dimensional space using FastMap for displaying and the refined distance metrics are 
mapped into an N-dimensional space using FastMap for redisplaying. 

(3) Jain discloses the claimed limitation that the first distance metrics are mapped into an 
N-dimensional space using FastMap for displaying and the refined distance metrics are mapped 
into an N-dimensional space using FastMap for redisplaying (e.g., Jain column 27, lines 30-40; 
column 25, lines 35-67; column 26, lines 1-57). 

(4) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Jain's mapping method into Hirata's method of visualizing and retrieving an image from the 
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database because Hirata suggests distance metric for image features with respect to image 
regions (Hirata column 4-5), 

(5) One having the ordinary skill in the art would have been motivated to do this because 
it would have advantageously provided the display of metric properties for representation in the 
display space (Jain column 25, lines 35-67; column 26, lines 1-57). 

10. Claim 15 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hirata et al. U.S. 
Patent No. 6,240,423 (hereinafter Hirata) as applied to claims 1 and 17 above, and further in 
view of Bates et al. U.S. Patent No. 5,528,259 (hereinafter Bates). 

11. Claim 15: 

(1) Hirata teaches a method of visualizing and retrieving a data file from a set of data 

files. 

(2) Hirata is silent on the claimed limitation of providing a display depth indication that 
represents an amount of overlapping of images on the display and scrolling to view images that 
were previously not viewable due to overlapping of the images. 

(3) Bates discloses the claimed limitation of providing a display depth indication that 
represents an amount of overlapping of images on the display and scrolling to view images that 
were previously not viewable due to overlapping of the images (e.g., Bates figures 4A-4C; 
column 4, lines 39-56; column 5, lines 25-45). 

(4) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Bates' scrolling of overlapping images and selecting of an alternate layer into Hirata's method of 
visualizing and retrieving an image from the database because Hirata suggests a user interface 
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selection of a portion of image and second level regions and objects (Hirata figures 1-11; column 
4). 

(5) One having the ordinary skill in the art would have been motivated to do this because 
it would have advantageously provided a pictorial representation of one of selected multiple 
layers of image data in a multi-dimensional scrolling method of manipulating or selecting image 
layers (e.g., Bates figures 4A-4C; column 4, lines 39-56; column 5, lines 25-45). 



Remarks 

12. Applicant's arguments, filed 04/28/2004, paper number 7, have been fully considered but 
they are not deemed to be persuasive. 

13. Applicant argues in essence with respect to the claim 1 and similar claims that: 

(A) "In contrast to the method disclosed and recited in Applicants' independent claims 1 
and 17, the Hirata patent fails to teach or suggest using a refined distance metric which is 
applied to a user selected subset of displayed images." 

In response to the arguments in (A), the Examiner asserts that Hirata teaches a refined 
metric which is applied to a user selected subset of displayed images. For example, in column 5- 
6, it is stated, "the calculation based on boundary line matching is applied to the search results of 
an initial search based upon region attributes... and outputting retrieved results." Hirata teaches 
using a distance metric such as a second similarity to retrieve images based on the set of similar 
images retrieved using a first similarity. The second similarity is now a "refined" distance metric 
because it refines the search for images. The second similarity of Hirata thus meets the claim 
limitation of "a refined distance metric." Here, a refined set of similar images is produced from 
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the first set of similar images produced using a first distance metric. See figure 9, 1 1 and 12; 
column 6-9. 

14. Applicant argues in essence with respect to the claim 1 and similar claims that: 

(B) "In addition, the Hirata patent fails to teach or suggest the claim 17 feature of 
interactive user selection for selecting a subset of images to be displayed. Rather, the 
query based system of the Hirata patent groups all images, and only creates new 
groupings based on a new query. The Hirata patent does not teach or suggest applying a 
second stage of image matching to a user selected subset of returned images " 
In response to the arguments in (B), the Examiner asserts that Hirata teaches applying a 
refined metric to a user-selected subset of displayed images. For example, in column 5-6, it is 
stated, "the calculation based on boundary line matching is applied to the search results of an 
initial search based upon region attributes... and outputting retrieved results" Hirata teaches using 
a distance metric such as a second similarity (e.g., boundary similarity) to retrieve images based 
on the set of similar images retrieved using a first similarity such as color similarity and/or shape 
similarity of a region. The second similarity is now a "refined" distance metric because it refines 
the search among the similar images. The second similarity of Hirata thus meets the claim 
limitation of "a refined distance metric" Here, a refined set of similar images is produced from 
the first set of similar images produced using a first distance metric. See figure 9, 1 1 and 12; 
column 6-9. 



15. 



Applicant argues in essence with respect to the claim 1 and similar claims that: 
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(C) "Hirata does not teach or suggest that a refined distance metric, as presently claimed, 
is used to redisplay the images of any defined region. . . As such, the Hirata patent fails to 
teach or suggest the user interactive selection feature of Applicant's claim 17." 
In response to the arguments in (C), the claim limitation calls for "redisplaying a portion 
of the images using a refined distance metric." Applicant's argument that "a refined distance 
metric. . . is used to redisplay the images of any defined region" is not exactly found in the claim. 
The Examiner asserts that Hirata teaches the claim limitation. For example, in column 5-6, it is 
stated, "the calculation based on boundary line matching is applied to the search results of an 
initial search based upon region attributes... and outputting retrieved results." Hirata teaches 
applying a refined metric to a user-selected subset of displayed images that has been retrieved 
using the region based matching. Hirata teaches using a distance metric such as a second 
similarity (e.g., boundary similarity) to retrieve images based on the set of similar images 
retrieved using a first similarity such as color similarity and/or shape similarity of a region. The 
second similarity is now a "refined" distance metric because it refines the search among the 
similar images. The second similarity of Hirata thus meets the claim limitation of "a refined 
distance metric." Here, a refined set of similar images is produced from the first set of similar 
images produced using a first distance metric and therefore is less than the first set of similar 
images. See figure 9, 1 1 and 12; column 6-9. As regards to the user interactive selection feature 
of the claim 17, Hirata teaches that a user can specify the regions of the images; e.g., the region 
division of figure 3; figures 1-11; column 6-9. 



16. 



Applicant argues in essence with respect to the claim 1 and similar claims that: 
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(D) "Hirata does not teach or suggest redisplaying a portion of the initial images using a 
refined distance metric." 

In response to the arguments in (D), the claim limitation calls for "redisplaying a portion 
of the images using a refined distance metric. The Examiner asserts that Hirata teaches the claim 
limitation. For example, in column 5-6, it is stated, "the calculation based on boundary line 
matching is applied to the search results of an initial search based upon region attributes... and 
outputting retrieved results." Hirata teaches applying a refined metric to a user-selected subset of 
displayed images that has been retrieved using the region based matching. Hirata teaches using a 
distance metric such as a second similarity (e.g., boundary similarity) to retrieve images based on 
the set of similar images retrieved using a first similarity such as color similarity and/or shape 
similarity of a region. The second similarity is now a "refined" distance metric because it refines 
the search among the similar images. The second similarity of Hirata thus meets the claim 
limitation of "a refined distance metric " Here, a refined set of similar images is produced from 
the first set of similar images produced using a first distance metric and therefore is less than the 
first set of similar images. See figure 9, 1 1 and 12; column 6-9. 



Conclusion 

17. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
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the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1. 136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

18. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (703) 605-1213. 
The examiner can normally be reached on 8:00 AM - 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (703) 305-4713. The fax phone numbers for the 
organization where this application or proceeding is assigned are (703) 308-6606 for regular 
communications and (703) 308-6606 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 395-3900. 



jew 



May 26, 2004 



miChael razavi 

SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 26U0 




